{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dxP3r4mvQRnX"
      },
      "source": [
        "##### Copyright 2024 Google LLC."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {
        "cellView": "form",
        "id": "uU6msr-hQZGj"
      },
      "outputs": [],
      "source": [
        "# @title Licensed under the Apache License, Version 2.0 (the \"License\");\n",
        "# you may not use this file except in compliance with the License.\n",
        "# You may obtain a copy of the License at\n",
        "#\n",
        "# https://www.apache.org/licenses/LICENSE-2.0\n",
        "#\n",
        "# Unless required by applicable law or agreed to in writing, software\n",
        "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
        "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
        "# See the License for the specific language governing permissions and\n",
        "# limitations under the License."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "HI6jQZyLQouk"
      },
      "source": [
        "# Gemma - 3 Function Calling: Local File Reader\n",
        "\n",
        "Google released Gemma-3, a new variant that comes with multimodality, multilingual support for up to 140 global languages, a 128K context window, and agentic capabilities like structured responses and function calling. In this notebook, we will explore how to use function calling with open-source models from Hugging Face via the Gemma 3 4B-it model to read and summarize local text files.\n",
        "\n",
        "<table align=\"left\">\n",
        "  <td>\n",
        "    <a target=\"_blank\" href=\"https://colab.research.google.com/github/google-gemini/gemma-cookbook/blob/main/Gemma/[Gemma_3]Function_Calling_with_HF_document_summarizer.ipynb\"><img src=\"https://www.tensorflow.org/images/colab_logo_32px.png\" />Run in Google Colab</a>\n",
        "  </td>\n",
        "</table>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8zdBd9FqRNYi"
      },
      "source": [
        "## Setup\n",
        "\n",
        "### Select the Colab runtime\n",
        "To complete this tutorial, you'll need to have a Colab runtime with sufficient resources to run the Gemma model. In this case, you can use a T4 GPU:\n",
        "\n",
        "1. In the upper-right of the Colab window, select **▾ (Additional connection options)**.\n",
        "2. Select **Change runtime type**.\n",
        "3. Under **Hardware accelerator**, select **T4 GPU**.\n",
        "\n",
        "### Gemma-3 setup\n",
        "\n",
        "To complete this tutorial, you'll first need to complete the setup instructions at [Gemma setup](https://ai.google.dev/gemma/docs/setup). The Gemma setup instructions show you how to do the following:\n",
        "\n",
        "* Get access to Gemma on kaggle.com.\n",
        "* Select a Colab runtime with sufficient resources to run the Gemma 3 4B-it model.\n",
        "* Generate and configure a Kaggle username and an API key as Colab secrets.\n",
        "\n",
        "After you've completed the Gemma setup, move on to the next section, where you'll set environment variables for your Colab environment.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "UPG9Tt-xRXnN"
      },
      "source": [
        "### Install dependencies\n",
        "\n",
        "Run the cell below to install all the required dependencies.\n",
        "\n",
        "- Transformers: Latest version that supports Gemma-3\n",
        "- Accelerate\n",
        "- No external APIs required; uses standard Python file I/O"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "metadata": {
        "id": "XpQ4yzBIHGbH"
      },
      "outputs": [],
      "source": [
        "!pip install git+https://github.com/huggingface/transformers.git"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 3,
      "metadata": {
        "id": "hX82hPriIM3Z"
      },
      "outputs": [],
      "source": [
        "!pip install accelerate"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "a_KJh4F1Rjav"
      },
      "source": [
        "# Gemma 3\n",
        "\n",
        "Google has released Gemma-3, a successor to its earlier Gemma models, focusing on open-source accessibility and multimodal vision capabilities. The model is designed to handle complex tasks with extended context lengths, support for multiple languages, and vision capabilities, making it suitable for a wide range of applications.\n",
        "\n",
        "Gemma-3 has gained attention for outperforming notable models like DeepSeek V3, o3-mini, and Llama 3 405B, with extended context lengths, multilingual support, and multimodal capabilities.\n",
        "\n",
        "- Extended context length from 32K to 128K using ROPE scaling (from 10K to 1M) on global self-attention layers.\n",
        "- Available in 1B, 4B, 12B, and 27B sizes (Vision capability included, except for 1B, with bi-directional attention).\n",
        "- Supports an extensive range of 140 global languages.\n",
        "- Additional Features: Supports function calling, where the model can generate code or instructions to call external functions, and structured outputs, enabling it to produce data in specific formats like JSON or tables."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xUSjniskUsyc"
      },
      "source": [
        "## Load Model weights"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 4,
      "metadata": {
        "id": "us9dCp87GgVN"
      },
      "outputs": [],
      "source": [
        "import torch\n",
        "from transformers import AutoProcessor, Gemma3ForConditionalGeneration"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 5,
      "metadata": {
        "id": "0p-2edQBR4af"
      },
      "outputs": [],
      "source": [
        "model_id = \"google/gemma-3-4b-it\""
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 6,
      "metadata": {
        "id": "LISsLCCiGUVm"
      },
      "outputs": [],
      "source": [
        "model = Gemma3ForConditionalGeneration.from_pretrained(\n",
        "    model_id, device_map=\"auto\"\n",
        ").eval()\n",
        "\n",
        "processor = AutoProcessor.from_pretrained(model_id)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 7,
      "metadata": {
        "id": "bnzOHRPtoQut"
      },
      "outputs": [],
      "source": [
        "torch.cuda.empty_cache()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "i47OYekFdM4V"
      },
      "source": [
        "## Function Calling Gemma\n",
        "\n",
        "Function calling is a feature that allows large language models to interact with external systems, APIs, and tools in a structured way. This feature enables LLMs to not just generate text responses but to invoke specific functions when appropriate based on user queries. In this notebook, we’ll use function calling to read and summarize a local text file uploaded to the Colab environment. For example, a user can ask the model to summarize a file named `notes.txt`."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 8,
      "metadata": {
        "id": "UgU2fCoycyhu"
      },
      "outputs": [],
      "source": [
        "def read_file(filename: str) -> str:\n",
        "    \"\"\"\n",
        "    Reads the contents of a local text file and returns it as a string.\n",
        "\n",
        "    Args:\n",
        "        filename: The name of the file to read (e.g., 'notes.txt')\n",
        "    Returns:\n",
        "        The contents of the file as a string, or an error message if the file is not found.\n",
        "    \"\"\"\n",
        "    try:\n",
        "        with open(filename, 'r', encoding='utf-8') as file:\n",
        "            return file.read()\n",
        "    except FileNotFoundError:\n",
        "        return f\"Error: File '{filename}' not found.\"\n",
        "    except Exception as e:\n",
        "        return f\"Error: {str(e)}\""
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 9,
      "metadata": {
        "id": "CQiUJlCBdb8Y"
      },
      "outputs": [],
      "source": [
        "tools = [read_file]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 10,
      "metadata": {
        "id": "R3z_9TbvufF3"
      },
      "outputs": [],
      "source": [
        "# Example query\n",
        "query = \"Summarize the contents of notes.txt\""
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "jHQrvzvRR7n8"
      },
      "source": [
        "## Define prompt for function calling\n",
        "\n",
        "Function Calling follows these steps:\n",
        "\n",
        "- Your application sends a prompt to the LLM along with function definitions\n",
        "- The LLM analyzes the prompt and decides whether to respond directly or use defined functions\n",
        "- If using functions, the LLM generates structured arguments for the function call\n",
        "- Your application receives the function call details and executes the actual function\n",
        "- The function results are sent back to the LLM\n",
        "- The LLM provides a final response incorporating the function results"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 11,
      "metadata": {
        "id": "j4LiCaOodjyn"
      },
      "outputs": [],
      "source": [
        "conversation = [\n",
        "    {\n",
        "        \"role\": \"system\",\n",
        "        \"content\": [{\"type\": \"text\", \"text\": \"\"\"\n",
        "        You are an expert file assistant. Use `read_file` to access and process local text files.\n",
        "        At each turn, if you decide to invoke any of the function(s), it should be wrapped with ```tool_code```. The Python methods described below are imported and available,\n",
        "        you can only use defined methods. The generated code should be readable and efficient. The response to a method will be wrapped in ```tool_output```; use it to call more tools or generate a helpful, friendly response.\n",
        "        When using a ```tool_call```, think step by step why and how it should be used.\n",
        "\n",
        "        The following Python methods are available:\n",
        "        ```python\n",
        "        def read_file(filename: str) -> str:\n",
        "          \"\n",
        "          Reads the contents of a local text file and returns it as a string.\n",
        "\n",
        "          Args:\n",
        "              filename: The name of the file to read (e.g., 'notes.txt')\n",
        "          Returns:\n",
        "              The contents of the file as a string, or an error message if the file is not found.\n",
        "          \"\n",
        "        ```\n",
        "\n",
        "        User: {user_message}\n",
        "        \"\"\"}]\n",
        "    },\n",
        "    {\n",
        "        \"role\": \"user\",\n",
        "        \"content\": [\n",
        "            {\"type\": \"text\", \"text\": query}\n",
        "        ]\n",
        "    }\n",
        "]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 12,
      "metadata": {
        "id": "nh51Dr6Xde9r"
      },
      "outputs": [],
      "source": [
        "inputs = processor.apply_chat_template(\n",
        "            conversation,\n",
        "            tools=tools,\n",
        "            add_generation_prompt=True,\n",
        "            return_dict=True,\n",
        "            tokenize=True,\n",
        "            return_tensors=\"pt\",\n",
        ").to(model.device, dtype=torch.bfloat16)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 13,
      "metadata": {
        "id": "xNO25997ebXH"
      },
      "outputs": [],
      "source": [
        "outputs = model.generate(**inputs, max_new_tokens=512)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 14,
      "metadata": {
        "id": "XN32I1J2ltjN"
      },
      "outputs": [],
      "source": [
        "output = processor.decode(outputs[0], skip_special_tokens=True)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 15,
      "metadata": {
        "id": "moOkqbxAo0TT"
      },
      "outputs": [],
      "source": [
        "response = output.split(\"model\")[-1]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 16,
      "metadata": {
        "id": "bGF5oJyntu7C"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "\n",
            "Okay, I can help with that. I will read the file \"notes.txt\" and provide a summary of its contents.\n",
            "\n",
            "```tool_code\n",
            "summary = read_file(filename='notes.txt')\n",
            "print(summary)\n",
            "```\n"
          ]
        }
      ],
      "source": [
        "print(response)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pldp3z7USS5P"
      },
      "source": [
        "## Execute the function calling code"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 20,
      "metadata": {
        "id": "ni57IK_OpIya"
      },
      "outputs": [],
      "source": [
        "import re\n",
        "import io\n",
        "from contextlib import redirect_stdout\n",
        "\n",
        "def extract_tool_call(text):\n",
        "    pattern = r\"```tool_code\\s*(.*?)\\s*```\"\n",
        "    match = re.search(pattern, text, re.DOTALL)\n",
        "    if match:\n",
        "        code = match.group(1).strip()\n",
        "        # Capture stdout in a string buffer\n",
        "        f = io.StringIO()\n",
        "        with redirect_stdout(f):\n",
        "            # result = eval(code)\n",
        "            result = exec(code, globals())\n",
        "        output = f.getvalue()\n",
        "        r = result if output == '' else output\n",
        "        return f'```tool_output\\n{r}\\n```'\n",
        "    return None"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 21,
      "metadata": {
        "id": "SRYVbLKdosaN"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "\n",
              "     <input type=\"file\" id=\"files-7fcbb965-676a-40e7-b541-a66f06e84660\" name=\"files[]\" multiple disabled\n",
              "        style=\"border:none\" />\n",
              "     <output id=\"result-7fcbb965-676a-40e7-b541-a66f06e84660\">\n",
              "      Upload widget is only available when the cell has been executed in the\n",
              "      current browser session. Please rerun this cell to enable.\n",
              "      </output>\n",
              "      <script>// Copyright 2017 Google LLC\n",
              "//\n",
              "// Licensed under the Apache License, Version 2.0 (the \"License\");\n",
              "// you may not use this file except in compliance with the License.\n",
              "// You may obtain a copy of the License at\n",
              "//\n",
              "//      http://www.apache.org/licenses/LICENSE-2.0\n",
              "//\n",
              "// Unless required by applicable law or agreed to in writing, software\n",
              "// distributed under the License is distributed on an \"AS IS\" BASIS,\n",
              "// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
              "// See the License for the specific language governing permissions and\n",
              "// limitations under the License.\n",
              "\n",
              "/**\n",
              " * @fileoverview Helpers for google.colab Python module.\n",
              " */\n",
              "(function(scope) {\n",
              "function span(text, styleAttributes = {}) {\n",
              "  const element = document.createElement('span');\n",
              "  element.textContent = text;\n",
              "  for (const key of Object.keys(styleAttributes)) {\n",
              "    element.style[key] = styleAttributes[key];\n",
              "  }\n",
              "  return element;\n",
              "}\n",
              "\n",
              "// Max number of bytes which will be uploaded at a time.\n",
              "const MAX_PAYLOAD_SIZE = 100 * 1024;\n",
              "\n",
              "function _uploadFiles(inputId, outputId) {\n",
              "  const steps = uploadFilesStep(inputId, outputId);\n",
              "  const outputElement = document.getElementById(outputId);\n",
              "  // Cache steps on the outputElement to make it available for the next call\n",
              "  // to uploadFilesContinue from Python.\n",
              "  outputElement.steps = steps;\n",
              "\n",
              "  return _uploadFilesContinue(outputId);\n",
              "}\n",
              "\n",
              "// This is roughly an async generator (not supported in the browser yet),\n",
              "// where there are multiple asynchronous steps and the Python side is going\n",
              "// to poll for completion of each step.\n",
              "// This uses a Promise to block the python side on completion of each step,\n",
              "// then passes the result of the previous step as the input to the next step.\n",
              "function _uploadFilesContinue(outputId) {\n",
              "  const outputElement = document.getElementById(outputId);\n",
              "  const steps = outputElement.steps;\n",
              "\n",
              "  const next = steps.next(outputElement.lastPromiseValue);\n",
              "  return Promise.resolve(next.value.promise).then((value) => {\n",
              "    // Cache the last promise value to make it available to the next\n",
              "    // step of the generator.\n",
              "    outputElement.lastPromiseValue = value;\n",
              "    return next.value.response;\n",
              "  });\n",
              "}\n",
              "\n",
              "/**\n",
              " * Generator function which is called between each async step of the upload\n",
              " * process.\n",
              " * @param {string} inputId Element ID of the input file picker element.\n",
              " * @param {string} outputId Element ID of the output display.\n",
              " * @return {!Iterable<!Object>} Iterable of next steps.\n",
              " */\n",
              "function* uploadFilesStep(inputId, outputId) {\n",
              "  const inputElement = document.getElementById(inputId);\n",
              "  inputElement.disabled = false;\n",
              "\n",
              "  const outputElement = document.getElementById(outputId);\n",
              "  outputElement.innerHTML = '';\n",
              "\n",
              "  const pickedPromise = new Promise((resolve) => {\n",
              "    inputElement.addEventListener('change', (e) => {\n",
              "      resolve(e.target.files);\n",
              "    });\n",
              "  });\n",
              "\n",
              "  const cancel = document.createElement('button');\n",
              "  inputElement.parentElement.appendChild(cancel);\n",
              "  cancel.textContent = 'Cancel upload';\n",
              "  const cancelPromise = new Promise((resolve) => {\n",
              "    cancel.onclick = () => {\n",
              "      resolve(null);\n",
              "    };\n",
              "  });\n",
              "\n",
              "  // Wait for the user to pick the files.\n",
              "  const files = yield {\n",
              "    promise: Promise.race([pickedPromise, cancelPromise]),\n",
              "    response: {\n",
              "      action: 'starting',\n",
              "    }\n",
              "  };\n",
              "\n",
              "  cancel.remove();\n",
              "\n",
              "  // Disable the input element since further picks are not allowed.\n",
              "  inputElement.disabled = true;\n",
              "\n",
              "  if (!files) {\n",
              "    return {\n",
              "      response: {\n",
              "        action: 'complete',\n",
              "      }\n",
              "    };\n",
              "  }\n",
              "\n",
              "  for (const file of files) {\n",
              "    const li = document.createElement('li');\n",
              "    li.append(span(file.name, {fontWeight: 'bold'}));\n",
              "    li.append(span(\n",
              "        `(${file.type || 'n/a'}) - ${file.size} bytes, ` +\n",
              "        `last modified: ${\n",
              "            file.lastModifiedDate ? file.lastModifiedDate.toLocaleDateString() :\n",
              "                                    'n/a'} - `));\n",
              "    const percent = span('0% done');\n",
              "    li.appendChild(percent);\n",
              "\n",
              "    outputElement.appendChild(li);\n",
              "\n",
              "    const fileDataPromise = new Promise((resolve) => {\n",
              "      const reader = new FileReader();\n",
              "      reader.onload = (e) => {\n",
              "        resolve(e.target.result);\n",
              "      };\n",
              "      reader.readAsArrayBuffer(file);\n",
              "    });\n",
              "    // Wait for the data to be ready.\n",
              "    let fileData = yield {\n",
              "      promise: fileDataPromise,\n",
              "      response: {\n",
              "        action: 'continue',\n",
              "      }\n",
              "    };\n",
              "\n",
              "    // Use a chunked sending to avoid message size limits. See b/62115660.\n",
              "    let position = 0;\n",
              "    do {\n",
              "      const length = Math.min(fileData.byteLength - position, MAX_PAYLOAD_SIZE);\n",
              "      const chunk = new Uint8Array(fileData, position, length);\n",
              "      position += length;\n",
              "\n",
              "      const base64 = btoa(String.fromCharCode.apply(null, chunk));\n",
              "      yield {\n",
              "        response: {\n",
              "          action: 'append',\n",
              "          file: file.name,\n",
              "          data: base64,\n",
              "        },\n",
              "      };\n",
              "\n",
              "      let percentDone = fileData.byteLength === 0 ?\n",
              "          100 :\n",
              "          Math.round((position / fileData.byteLength) * 100);\n",
              "      percent.textContent = `${percentDone}% done`;\n",
              "\n",
              "    } while (position < fileData.byteLength);\n",
              "  }\n",
              "\n",
              "  // All done.\n",
              "  yield {\n",
              "    response: {\n",
              "      action: 'complete',\n",
              "    }\n",
              "  };\n",
              "}\n",
              "\n",
              "scope.google = scope.google || {};\n",
              "scope.google.colab = scope.google.colab || {};\n",
              "scope.google.colab._files = {\n",
              "  _uploadFiles,\n",
              "  _uploadFilesContinue,\n",
              "};\n",
              "})(self);\n",
              "</script> "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Saving notes.txt to notes (2).txt\n"
          ]
        },
        {
          "data": {
            "text/html": [
              "\n",
              "     <input type=\"file\" id=\"files-0d6bdfbe-d2ef-4535-98e1-f4cb8ddfc958\" name=\"files[]\" multiple disabled\n",
              "        style=\"border:none\" />\n",
              "     <output id=\"result-0d6bdfbe-d2ef-4535-98e1-f4cb8ddfc958\">\n",
              "      Upload widget is only available when the cell has been executed in the\n",
              "      current browser session. Please rerun this cell to enable.\n",
              "      </output>\n",
              "      <script>// Copyright 2017 Google LLC\n",
              "//\n",
              "// Licensed under the Apache License, Version 2.0 (the \"License\");\n",
              "// you may not use this file except in compliance with the License.\n",
              "// You may obtain a copy of the License at\n",
              "//\n",
              "//      http://www.apache.org/licenses/LICENSE-2.0\n",
              "//\n",
              "// Unless required by applicable law or agreed to in writing, software\n",
              "// distributed under the License is distributed on an \"AS IS\" BASIS,\n",
              "// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
              "// See the License for the specific language governing permissions and\n",
              "// limitations under the License.\n",
              "\n",
              "/**\n",
              " * @fileoverview Helpers for google.colab Python module.\n",
              " */\n",
              "(function(scope) {\n",
              "function span(text, styleAttributes = {}) {\n",
              "  const element = document.createElement('span');\n",
              "  element.textContent = text;\n",
              "  for (const key of Object.keys(styleAttributes)) {\n",
              "    element.style[key] = styleAttributes[key];\n",
              "  }\n",
              "  return element;\n",
              "}\n",
              "\n",
              "// Max number of bytes which will be uploaded at a time.\n",
              "const MAX_PAYLOAD_SIZE = 100 * 1024;\n",
              "\n",
              "function _uploadFiles(inputId, outputId) {\n",
              "  const steps = uploadFilesStep(inputId, outputId);\n",
              "  const outputElement = document.getElementById(outputId);\n",
              "  // Cache steps on the outputElement to make it available for the next call\n",
              "  // to uploadFilesContinue from Python.\n",
              "  outputElement.steps = steps;\n",
              "\n",
              "  return _uploadFilesContinue(outputId);\n",
              "}\n",
              "\n",
              "// This is roughly an async generator (not supported in the browser yet),\n",
              "// where there are multiple asynchronous steps and the Python side is going\n",
              "// to poll for completion of each step.\n",
              "// This uses a Promise to block the python side on completion of each step,\n",
              "// then passes the result of the previous step as the input to the next step.\n",
              "function _uploadFilesContinue(outputId) {\n",
              "  const outputElement = document.getElementById(outputId);\n",
              "  const steps = outputElement.steps;\n",
              "\n",
              "  const next = steps.next(outputElement.lastPromiseValue);\n",
              "  return Promise.resolve(next.value.promise).then((value) => {\n",
              "    // Cache the last promise value to make it available to the next\n",
              "    // step of the generator.\n",
              "    outputElement.lastPromiseValue = value;\n",
              "    return next.value.response;\n",
              "  });\n",
              "}\n",
              "\n",
              "/**\n",
              " * Generator function which is called between each async step of the upload\n",
              " * process.\n",
              " * @param {string} inputId Element ID of the input file picker element.\n",
              " * @param {string} outputId Element ID of the output display.\n",
              " * @return {!Iterable<!Object>} Iterable of next steps.\n",
              " */\n",
              "function* uploadFilesStep(inputId, outputId) {\n",
              "  const inputElement = document.getElementById(inputId);\n",
              "  inputElement.disabled = false;\n",
              "\n",
              "  const outputElement = document.getElementById(outputId);\n",
              "  outputElement.innerHTML = '';\n",
              "\n",
              "  const pickedPromise = new Promise((resolve) => {\n",
              "    inputElement.addEventListener('change', (e) => {\n",
              "      resolve(e.target.files);\n",
              "    });\n",
              "  });\n",
              "\n",
              "  const cancel = document.createElement('button');\n",
              "  inputElement.parentElement.appendChild(cancel);\n",
              "  cancel.textContent = 'Cancel upload';\n",
              "  const cancelPromise = new Promise((resolve) => {\n",
              "    cancel.onclick = () => {\n",
              "      resolve(null);\n",
              "    };\n",
              "  });\n",
              "\n",
              "  // Wait for the user to pick the files.\n",
              "  const files = yield {\n",
              "    promise: Promise.race([pickedPromise, cancelPromise]),\n",
              "    response: {\n",
              "      action: 'starting',\n",
              "    }\n",
              "  };\n",
              "\n",
              "  cancel.remove();\n",
              "\n",
              "  // Disable the input element since further picks are not allowed.\n",
              "  inputElement.disabled = true;\n",
              "\n",
              "  if (!files) {\n",
              "    return {\n",
              "      response: {\n",
              "        action: 'complete',\n",
              "      }\n",
              "    };\n",
              "  }\n",
              "\n",
              "  for (const file of files) {\n",
              "    const li = document.createElement('li');\n",
              "    li.append(span(file.name, {fontWeight: 'bold'}));\n",
              "    li.append(span(\n",
              "        `(${file.type || 'n/a'}) - ${file.size} bytes, ` +\n",
              "        `last modified: ${\n",
              "            file.lastModifiedDate ? file.lastModifiedDate.toLocaleDateString() :\n",
              "                                    'n/a'} - `));\n",
              "    const percent = span('0% done');\n",
              "    li.appendChild(percent);\n",
              "\n",
              "    outputElement.appendChild(li);\n",
              "\n",
              "    const fileDataPromise = new Promise((resolve) => {\n",
              "      const reader = new FileReader();\n",
              "      reader.onload = (e) => {\n",
              "        resolve(e.target.result);\n",
              "      };\n",
              "      reader.readAsArrayBuffer(file);\n",
              "    });\n",
              "    // Wait for the data to be ready.\n",
              "    let fileData = yield {\n",
              "      promise: fileDataPromise,\n",
              "      response: {\n",
              "        action: 'continue',\n",
              "      }\n",
              "    };\n",
              "\n",
              "    // Use a chunked sending to avoid message size limits. See b/62115660.\n",
              "    let position = 0;\n",
              "    do {\n",
              "      const length = Math.min(fileData.byteLength - position, MAX_PAYLOAD_SIZE);\n",
              "      const chunk = new Uint8Array(fileData, position, length);\n",
              "      position += length;\n",
              "\n",
              "      const base64 = btoa(String.fromCharCode.apply(null, chunk));\n",
              "      yield {\n",
              "        response: {\n",
              "          action: 'append',\n",
              "          file: file.name,\n",
              "          data: base64,\n",
              "        },\n",
              "      };\n",
              "\n",
              "      let percentDone = fileData.byteLength === 0 ?\n",
              "          100 :\n",
              "          Math.round((position / fileData.byteLength) * 100);\n",
              "      percent.textContent = `${percentDone}% done`;\n",
              "\n",
              "    } while (position < fileData.byteLength);\n",
              "  }\n",
              "\n",
              "  // All done.\n",
              "  yield {\n",
              "    response: {\n",
              "      action: 'complete',\n",
              "    }\n",
              "  };\n",
              "}\n",
              "\n",
              "scope.google = scope.google || {};\n",
              "scope.google.colab = scope.google.colab || {};\n",
              "scope.google.colab._files = {\n",
              "  _uploadFiles,\n",
              "  _uploadFilesContinue,\n",
              "};\n",
              "})(self);\n",
              "</script> "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Saving notes.txt to notes (3).txt\n"
          ]
        }
      ],
      "source": [
        "# Upload a sample file to Colab first (e.g., notes.txt)\n",
        "from google.colab import files\n",
        "uploaded = files.upload()\n",
        "\n",
        "# Extract tool call from the response\n",
        "keyword = extract_tool_call(response)\n",
        "# Upload a sample file to Colab first (e.g., notes.txt)\n",
        "from google.colab import files\n",
        "uploaded = files.upload()\n",
        "\n",
        "# Extract tool call from the response\n",
        "keyword = extract_tool_call(response)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 22,
      "metadata": {
        "id": "eznPlrIBpddw"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            },
            "text/plain": [
              "'```tool_output\\nAdapt AI :A personalized solution to sense your stress, fix your mess, and , boost productivity.\\nMaintaining productivity while managing stress is a significant challenge in todays fast paced and multi tasking work environment.\\n\\nConventional tools are often static in nature, and offer generic solutions that fail to consider the nuances of individual differences in behavior, and stress triggers.\\n\\nTherefore in this Late breaking work we introduce Adapt AI: a groundbreaking, personalized AI solution designed to boost productivity and enhance well-being through tailored, personalized, interventions.\\n\\nAdapt AI employs a sophisticated multimodal sensing system, integrating egocentric vision, audio, and physiological data such as heart and motion activities of the individuals.\\n\\nThis allows Adapt AI to understand and respond to the unique context of each user with the help of Large language models.\\n\\nBy monitoring real-time data, Adapt AI detects stress levels and activity patterns, dynamically suggesting interventions like micro-breaks or physical activities precisely when needed.\\n\\nThe core of Adapt AI includes several critical components:\\nThe Processing Module: This Captures and processes inputs from cameras, microphones, and physiological sensors and feeds the data to LLMs, to create a comprehensive understanding of the user’s current state.\\n\\nThen, the, External Task Agents, Automate routine tasks, allowing users to focus on more critical responsibilities.\\n\\nThe Personalized Well-being Intervention Pipeline Uses insights generated from the data to suggest timely actions that help maintain a healthy work rhythm and reduce stress.\\n\\nUsing all the above our system is able to sense your stress and fix your mess, but thats not all, we also boost productivity using your own personalized Tone adaptive agent.\\n\\nThe Tone-Adaptive Conversation Agent is a personalized LLM at your service that answers to your day to day queries while adjusting its communication style based on the user’s emotional state, offering support and suggestions through a user-friendly interface.\\n\\nFor instance, if the system detects increased stress during a complex project, it might suggest a brief relaxation exercise or a change in task focus to help alleviate stress, while also acting cheerful and motivational in the form of its conversational agent responses.\\n\\nIn practice, Adapt AI functions as a smart solution that not only manages tasks but also proactively supports the user’s mental and physical health through personalized assistance.\\n\\nIn a study with 15 participants, Our AdaptAI demonstrated significant improvements in task performance and user satisfaction, validating its effectiveness in creating a more supportive and efficient work environment.\\n\\nThis intuitive system not only addresses immediate needs but also adapts to changing conditions, ensuring optimal performance and well-being in any professional setting. Thank you!\\n\\n\\n\\n```'"
            ]
          },
          "execution_count": 22,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "keyword"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 23,
      "metadata": {
        "id": "Ek9wlj3xuOnc"
      },
      "outputs": [],
      "source": [
        "prompt = [{\n",
        "        \"role\": \"user\",\n",
        "        \"content\": [\n",
        "            {\"type\": \"text\", \"text\": f\"Results from the file read:{keyword} User_query: {query}\"}\n",
        "        ]\n",
        "    }\n",
        "]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 24,
      "metadata": {
        "id": "QB331y_TvN7K"
      },
      "outputs": [],
      "source": [
        "final_prompt = processor.apply_chat_template(\n",
        "            prompt,\n",
        "            tools=tools,\n",
        "            add_generation_prompt=True,\n",
        "            return_dict=True,\n",
        "            tokenize=True,\n",
        "            return_tensors=\"pt\",\n",
        ").to(model.device, dtype=torch.bfloat16)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 25,
      "metadata": {
        "id": "rcjE05aSvHEI"
      },
      "outputs": [],
      "source": [
        "response = model.generate(**final_prompt, max_new_tokens=512)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 26,
      "metadata": {
        "id": "aBcgbJjEvlW4"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "user\n",
            "Results from the file read:```tool_output\n",
            "Adapt AI :A personalized solution to sense your stress, fix your mess, and , boost productivity.\n",
            "Maintaining productivity while managing stress is a significant challenge in todays fast paced and multi tasking work environment.\n",
            "\n",
            "Conventional tools are often static in nature, and offer generic solutions that fail to consider the nuances of individual differences in behavior, and stress triggers.\n",
            "\n",
            "Therefore in this Late breaking work we introduce Adapt AI: a groundbreaking, personalized AI solution designed to boost productivity and enhance well-being through tailored, personalized, interventions.\n",
            "\n",
            "Adapt AI employs a sophisticated multimodal sensing system, integrating egocentric vision, audio, and physiological data such as heart and motion activities of the individuals.\n",
            "\n",
            "This allows Adapt AI to understand and respond to the unique context of each user with the help of Large language models.\n",
            "\n",
            "By monitoring real-time data, Adapt AI detects stress levels and activity patterns, dynamically suggesting interventions like micro-breaks or physical activities precisely when needed.\n",
            "\n",
            "The core of Adapt AI includes several critical components:\n",
            "The Processing Module: This Captures and processes inputs from cameras, microphones, and physiological sensors and feeds the data to LLMs, to create a comprehensive understanding of the user’s current state.\n",
            "\n",
            "Then, the, External Task Agents, Automate routine tasks, allowing users to focus on more critical responsibilities.\n",
            "\n",
            "The Personalized Well-being Intervention Pipeline Uses insights generated from the data to suggest timely actions that help maintain a healthy work rhythm and reduce stress.\n",
            "\n",
            "Using all the above our system is able to sense your stress and fix your mess, but thats not all, we also boost productivity using your own personalized Tone adaptive agent.\n",
            "\n",
            "The Tone-Adaptive Conversation Agent is a personalized LLM at your service that answers to your day to day queries while adjusting its communication style based on the user’s emotional state, offering support and suggestions through a user-friendly interface.\n",
            "\n",
            "For instance, if the system detects increased stress during a complex project, it might suggest a brief relaxation exercise or a change in task focus to help alleviate stress, while also acting cheerful and motivational in the form of its conversational agent responses.\n",
            "\n",
            "In practice, Adapt AI functions as a smart solution that not only manages tasks but also proactively supports the user’s mental and physical health through personalized assistance.\n",
            "\n",
            "In a study with 15 participants, Our AdaptAI demonstrated significant improvements in task performance and user satisfaction, validating its effectiveness in creating a more supportive and efficient work environment.\n",
            "\n",
            "This intuitive system not only addresses immediate needs but also adapts to changing conditions, ensuring optimal performance and well-being in any professional setting. Thank you!\n",
            "\n",
            "\n",
            "\n",
            "``` User_query: Summarize the contents of notes.txt\n",
            "model\n",
            "Okay, here's a summary of the provided text about Adapt AI:\n",
            "\n",
            "**Adapt AI is a personalized AI solution designed to boost productivity and enhance well-being in a fast-paced work environment.** It addresses the limitations of traditional tools by recognizing individual differences and stress triggers.\n",
            "\n",
            "**Here's how it works:**\n",
            "\n",
            "*   **Sensing:** It uses a multimodal system – cameras, microphones, and physiological sensors (like heart rate and motion) – to understand the user’s context and emotional state.\n",
            "*   **Processing:** The data is fed into Large Language Models (LLMs) to create a comprehensive understanding of the user’s current state.\n",
            "*   **Task Automation:** External Task Agents automate routine tasks, freeing up the user’s time.\n",
            "*   **Personalized Interventions:**  A Well-being Intervention Pipeline suggests timely actions like micro-breaks or physical activities based on detected stress levels.\n",
            "*   **Tone-Adaptive Conversation Agent:** A personalized LLM adjusts its communication style to the user’s emotional state, offering support and motivation. For example, it can suggest relaxation exercises during stressful times or provide cheerful encouragement.\n",
            "\n",
            "**Results:** A study with 15 participants showed significant improvements in task performance and user satisfaction, validating Adapt AI's effectiveness.\n",
            "\n",
            "**In essence, Adapt AI proactively monitors the user, adjusts its approach based on their needs, and provides support for both productivity and mental/physical well-being.**\n",
            "\n",
            "---\n",
            "\n",
            "Do you want me to focus on a specific aspect of the summary, or perhaps extract particular information (like the components of the system)?\n"
          ]
        }
      ],
      "source": [
        "print(processor.decode(response[0], skip_special_tokens=True))"
      ]
    }
  ],
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "name": "[Gemma_3]Function_Calling_with_HF_document_summarizer.ipynb",
      "toc_visible": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}
